Optimizing Anthrax Outbreak Detection Using Reinforcement Learning

The potentially catastrophic impact of a bioterrorist attack makes developing effective detection methods essential for public health. In the case of anthrax attack, a delay of hours in making a right decision can lead to hundreds of lives lost. Current detection methods trade off reliability of alarms for early detection of outbreaks. The performance of these methods can be improved by modern disease-specific modeling techniques which take into account the potential costs and effects of an attack to provide optimal warnings. We study this optimization problem in the reinforcement learning framework. The key contribution of this paper is to apply Partially Observable Markov Decision Processes (POMDPs) on outbreak detection mechanism for improving alarm function in anthrax outbreak detection. Our approach relies on estimating the future benefit of true alarms and the costs of false alarms and using these quantities to identify an optimal decision. We present empirical evidence illustrating that the performance of detection methods with respect to sensitivity and timeliness is improved significantly by utilizing POMDPs.